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PACS 05. 40. -a - Fluctuation phenomena, random processes, noise, and Brownian motion 
PACS 05 . 70 . Ln - Nonequilibrium and irreversible thermodynamics 

Abstract. - The steady state of a Langevin equation with short ranged memory and coloured 
noise is analyzed. When the fluctuation-dissipation theorem of second kind is not satisfied, the 
dynamics is irreversible, i.e. detailed balance is violated. We show that the entropy production 
rate for this system should include the power injected by "memory forces". With this additional 
contribution, the Fluctuation Relation is fairly verified in simulations. Both dynamics with in- 
ertia and overdamped dynamics yield the same expression for this additional power. The role of 
"memory forces" within the fluctuation-dissipation relation of first kind is also discussed. 



Introduction. — Irreversibility of path probabilities 
accompanies the lack of thermodynamic equilibrium. This 
fact, which is obvious in relaxing systems, can also be 
shown in many models of statistically steady states. In 
the latter case there is the evidence for a general con- 
nection between irreversibility and power dissipated by 
non-conservative forces. A paradigm of this situation is 
a particle performing Brownian motion under the action 
of conservative and non-conservative forces: 



-TV- VC/(x)+F„ c [x(t)]+T7, 



(1) 



X 



with r\ a Gaussian white noise, (rj) = and (i]i(t)rij (t')} = 
2"fTSijS(t — t'), where i, j are component indices. It is 
easily shown that 



W t = log 



^({y(*m) 



c(s)] • v(s)ds. (2) 



where {Iv(s)}g is the time- reversed trajectory. Wt is usu- 
ally called "produced entropy" [1-3], and in some deter- 
ministic models with certain assumptions it is equivalent, 
neglecting finite differences, to the time-integral of the 
phase space contraction rate [4-6] . 

A phenomenon which may be enhanced in statistically 
steady states under the effect of non-conservative forces 
and/or multiple baths is that of correlations among de- 
grees of freedom: if one insists on a reduced description of 
the system, such as the motion of a tracer particle, these 
correlations must be reintroduced as retarded feedback, or 



memory, accompanied by colored noise: 

- 7 v - - f 7 (i - t')v(t')dt' (3) 

J — GO 

(Vi(t)Vj(t')) -^M^t-t'). (4) 

We stress the fact that memory acts in equilibrated 
systems too, as extensively discussed in classical refer- 
ences [7], where the condition for thermal equilibrium, 
in the absence of non-conservative forces, is given by the 
fluctuation-dissipation relation of the second kind 



M ij {t-t')=T5 ijl (\t-t'\), 



(5) 



which guarantees: (a) canonical distribution, (b) equipar- 
tition between the particle and the surrounding fluid, (c) 
the validity of the fluctuation-dissipation relation (FDR) 
of the first kincfl [8] 



6v(t) 
8v(0) 



(6) 



Violations of Eq. ^ or ([5]) are associated with lack of 
equilibrium, i.e. presence of more than one thermostat 
and/or of non-conservative external forces. This is, for 
instance, the case of a granular liquid, that is a gas of 
inelastic hard particles coupled to an external thermostat 
and at a packing fraction between 20% and 50%: in this 
system it appears that particles are correlated among each 



1 where 7 and (•) denote perturbed and unperturbed averages, re- 
spectively. 
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others and this is reflected in a non-Markovian dynamics 
for a tracer, together with a violation of the Einstein rela- 
tion [9-11]. A similar case is realized in a molecular liquid 
coupled to different thermostats, where the motion of a 
massive tracer is described by a Langevin equation with 
(short ranged) memory, and Eq. is not satisfied [12]. 

Entropy production for Langevin models with short- 
ranged memory has been recently studied in [13-16]. In 
the first three papers [13-15], the steady state is treated 
under the assumption of validity of ([5]), called "equilib- 
rium" bath, so that the irreversibility takes again the form 
of work done by external non-conservative forces, similar 
to Eq. In [14] a non-transparent general formula, for 
the case where is violated, is also given, involving a 
triple time-convolution between forces, memory j(t) and 
the inverse of noise color M(t). In [16], irreversibility of 
paths is given as a convolution of power dissipated by all 
(non-conservative and conservative) forces with a time- 
dependent "effective temperature" T e ff(t) which charac- 
terizes the violation of j5]). In the formulation of [16], the 
entropy production vanishes if both anharmonic potential 
and non-conservative external forces are absent. 

In this letter we show, for rapidly decaying memory ker- 
nels with different time-scales, within a class of possible 
violations of Eq. (O, a simple formula, Eq. (|3"5"|) . for the 
entropy production: in our formula memory appears as a 
force performing work on the system, if and only if rela- 
tion ([5]) is not satisfied. The fact that memory alone pro- 
duces entropy has not been explicitly discussed in other 
references: this discrepancy is likely to be due to a differ- 
ent definition for entropy production. We consider the one 
given by Lebowitz and Spohn [2], which is related to the 
time-derivative of the Gibbs Entropy and which is prop- 
erly derived if the dynamics is Markovian, i.e. all degrees 
of freedom required to determine the future are known in 
the present. To this purpose we consider a class of non- 
Markovian dynamics which can be mapped to Markovian 
systems by introducing auxiliary degrees of freedom. An 
analogous approach is adopted in [11] in order to give an 
interpretation of "violations" of FDR. 

It could be argued that our formula includes contri- 
butions to the entropy production coming from "internal 
forces", which is not physical in a steady state. On the 
contrary, we stress the general fact that memory cannot 
exist for an isolated particle: stated differently, the effect 
of past history influences the present only if other degrees 
of freedom (left out from the description) "remind" it to 
the particle. Recollisions in dense fluids constitute a typi- 
cal mechanism [17]. Therefore memory should be seen as 
an "external" but hidden force. In the models considered 
here, Eqs. (|24|) and (f39|) . this role is made explicit when 
the auxiliary variables arc introduced. We stress the fact 
that, since only steady states are investigated here, cases 
with long-range memory are not analyzed [18]. 

The plan of the paper is the following: we first dis- 
cuss our general formula, and give details for the inertia! 
and overdamped dynamics of a tracer, showing that the 



Fluctuation Relation [1,2] is verified for entropy produc- 
tion when the memory contribution is taken into account; 
then we analyze the connection between memory forces 
and the violation of FDR, previously studied in [11]; fi- 
nally we draw conclusions and perspectives. 

Irreversibility of multivariate paths. Let us 

first discuss a non-multiplicative multivariate Langevin 
equation with N degrees of freedom and without memory: 



^< = D<(X)+^(t) > 



(7) 



with i G [0, N — 1], £j(t) is a Gaussian process with 
(&(*)} = and = 2SijS(t - t'). The proba- 

bility distribution /t(X) satisfies 



at 



E 



dSjjX) 



(8) 



with the probability current defined by 

St(X) = A(X)/ t (X) - J2 S^ D vM x ) ( 9 ) 

3 3 

where Dij = y\ gikSjk is symmetric by construction. 

Variables Xi are assumed to have a well-defined par- 
ity €i — ±1, with respect to time- reversal. This leads to 
recognize reversible and irreversible parts of the drift: 



D i (X) = D r i ev (X) + Di r (K) 



(10) 



with 



Dr (X) = -[A(X) - £i A(eX)] = -e.DrieX) (11) 
A r (X) = i[A(X) + e;A(eX)] = e^TieX) (12) 

having defined eX = (e A , ei-X"i, ...e^_iX^_i). This de- 
composition can be extended to the probability current: 

dft 



ST 



Si 



dX 



j 

(13) 



Following classical references [19,20], the expression for 
conditional path probability of trajectory {X(s)}g, i^| 



logP({X(s)Y ) = -W 

jk 



dsDj^X^-D^Xis)}} 



x {X k (s) - D k [X(s)}}, (14) 



where we have assumed that D-- 1 exists. 

. . 3 

By using Eq. <)10|) and a few passages we get 



W t = log 



p({ix( s )Y ) 



XjXk + DJD l k r 



1 f 

2 Jo ]k 

D rev D rev _ 2 X.£)™ 



-2a 



jk 



DJX k - DJD r k e 



}, (15) 



2 We use the Ito convention for stochastic integrals. Note that, 
with this convention, the Jacobian in the path probability is 1 [21]. 
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where we have introduced the following definitions: 
l-e,e fc , l + e 7 e fe 



with 



(16) 



u jk - 2 jk ~ 2 

Eq. (|15p is strongly simplified in the case of a diagonal 
diffusion matrix Z),-,-, obtaining: 



E^ 1 









v- nre?; 







(17) 



We are not aware of any previous derivation of this for- 
mula in the literature. It is not difficult to generalize it to 
multiplicative processes (i.e. D t j dependent upon X). 

We now recall that, in order to obtain the complete 
path probability in the steady state, one has to multiply 
P({X(s)} ) by /(X(0)), where / = lim f t is the sta- 

t — >+oo 

tionary probability distribution. It is therefore possible to 
compute a different quantity 

/[X(0)]P({X( s )}<) 



wi 



log 



ir. 



/[eX(f)]P({ZX(*)}*) 
& t = log{/[X(0)]}-log{/[eX(t)]}. 



(18) 
(19) 



The term bt is usually known as "border term" [22]. 

It is interesting also to define "entropy production rates" 
cr and er', such that W t = J <r(s)ds and W[ = J Q a'(s)ds. 
The following decomposition can be done: 

a= -| (log/) + <7i + (T 2 , o' = o-i+ o 2 (20) 
at 



i x 

<T2 = -J2 D uDt r Dr. 



(21) 
(22) 



This decomposition will be particularly useful in the last 
section, when discussing the connection with the FDR. 

The condition of detailed balance is equivalent to a' = 
for all trajectories in the steady state. When detailed 
balance does not hold, it can be shown that 



log 



P(WI = x) 
p(W{ = -x) 



(23) 



where p(W[ = x) is the probability in the steady state. 
Eq. (|23|) is the finite-time Fluctuation Relation (FR). In 
general, excluding cases discussed in the literature [22-25] 
for large times t one has Wt 
satisfied by W t [1-3]. 



W[ and relation (|23|) is also 



Memory through auxiliary variables. The first 
class of generalized Langevin equations with memory we 
are interested in, concerns inertial dynamics, i.e. we con- 
sider the following equation of motion (we restrict our- 
selves to the one-dimensional problem, without loss of gen- 
erality): 



x = v 



7 (t) = 2 7o 6(t) 



M 
1 1 



( V (t))=0 (25) 



(26) 



and where F is a generic drift term which can take the 
form of a sum of conservative and non-conservative forces, 
i.e. F(x) = + F nc {x). When T { = T Q for all i, the 

FDR of the second kind, Eq. ([5]), holds. This model has 
several applications: among others, it has been proposed 
in [12] for weakly driven glassy systems; the dynamics of a 
tracer particle in moderately dense fluidized granular me- 
dia, including its linear response properties, are consistent 
with this model [9], and, recently, the noise in feedback 
cooled oscillator for gravitational wave detectors [26] has 
been characterized in a similar fashion [27] Q 

It is useful to map Eq. ([24]) into Eq. (J7]) , where all noises 
are uncorrelated, identifying N = M + 2 and Xo = v, 
Xjv-i = x, while Xi = Vi (i € [l,Af]) are auxiliary vari- 
ables necessary to take into account memory, for instance 
they can be defined as 



v{t') + x i^-Ut')\dt'. (27) 



With this choice for the auxiliary variables, it is easy to 
verify that the drifts in Eq. ([7]) are 




M 



F(x) - 7o« - ^2 \ ~ l 



1 



Vi (ie[i,M\) 



A = 

while the diagonal diffusion matrix reads 
£>oo = 7o ?o Di 



T 

Tj 



(28) 

(29) 
(30) 

(31) 



Summarizing, the system with memory is recast into a 
system of (linearly) coupled Langevin equations where all 
noises are uncorrelated. Auxiliary variables Vi (i £ [1, M]) 
are even under time-reversal, i.e. e; = 1 for i > 0: this 
can be understood, for instance, requiring the validity of 
detail balance in the equilibrium case T, = To for all i. 
Then we obtain 



r^rev 



U N-1 




TV 

Vi 



-lov 



(i€[l,M}) 



1 = 0. 



(32) 

(33) 
(34) 



3 for the models presented here, this is the only relevant case. 



4 Note that the pairing of equal characteristic times for the expo- 
nentials in (1 25 I t and H26H is not so restrictive: indeed, case Tj = 
or case fi — > 0, Tj — » oo with finite 7iTj, for some i, can be easily 
worked out and make no exception to the following analysis. 
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When computing Eq. (fP7|) . it is crucial to note that 
is not positive definite and cannot be inverted. Anyway, 
as noted by Machlup and Onsager [28], the last row and 
column of Dij (those associated to variable x, which has 
not explicit noise dependence) can be dropped out for the 
purpose of computing path probabilities. With this obser- 
vation, formula (fT7|) can be used, leading to 



W, 



y> S(vf) su 

F nc [x(s)]+Y^Fi[vi(s)}) v(s)ds 




(35) 



(36) 



As usual, exact differences appear, denoted as 6(g) = 
g(t) — g(Q). The non-trivial part of Wt is the time-integral 
on the r.h.s. of (|35|) . which does not reduce to exact dif- 
ferences: it is equivalent to the work done by the usual 
non-conservative external force F nc (x) and by new forces 
Fi expressed in Eq. (|36[> . It can be verified that memory 
forces do not depend on the definition of auxiliary vari- 
ables, Eq. (|27[) . as expected. The additional work done by 
forces Fi is due to feedback of past history on the parti- 
cle velocity and it is interesting to discover its effect on 
irreversibility. From formula (|35p it is also evident that 
force Fi vanishes if Tj = T . If Tj = T for all i, then the 
second kind FDR, Eq. ([5]), holds, and memory does not 
contribute to W t . 

We conclude this section, evaluating the so-called finite 
time (or transient) contribution b t which must be added in 
order to verify the FR at short times [22] , in the particular 
case M = 1 (only one auxiliary variable). We consider 
a setup where there is no external force, F — 0, apart 
from two reflecting walls confining the particle to have a 
uniform spatial distribution in between, making irrelevant 
the contribution of x to the stationary probability density. 
Considering only variables v = (vq = v,vi), one has: 



/(v) oc exp f-i J2 >; , ; ' r '''/j 



(37) 



where the inverse covariance matrix elements read Sqq 1 
^ (l + n AT) , ^ = -L (l - 2^ At) and S^ 1 



^VtFT^^AT, with AT = T — Ti, T* = 70^+^ 



Q 

and Q = (T*) 2 



^T T 1 (l+ 7o r 1 ) 2 . In summary: 



J oo 



-11 



Soi 1 [«i (0)v(0) +vi(t)v(t)] (38) 



Let us note that, when T = T\ (validity of the FDR of 
the second kind) , the exact difference (first term) of W t ex- 
actly cancels bt'. in particular, if external non-conservative 
forces are absent, it appears that W[ = 0, i.e. detailed bal- 
ance is satisfied. 
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Fig. 1: A: pdf of Wt for the inertial Langevin equation (|24|l 
with simple exponential memory, i.e. M = 1, Tq = 0.6, T\ — 3, 
70 = 10, ri — 10, 71 = 5, and F = 0, for different times 
f of integration. B: at large times, \ogp(Wt/t)/t converges 
to a time-independent function: the large deviation rate. C- 
D: check of the Fluctuation Relation (FR), Eq. ((23")) which is 
verified for all data aligned along the bisector. At small time 
(empty circles), where the FR does not hold, Eq. (|23[) is verified 
for the pdf of W[ (gray circles). Note that the only source of 
entropy, for this example, is memory. 



In Figure [T] we show the probability density function 
(pdf) p(Wt), in the steady state, obtained numerically by 
integrating Eq. for different choices of times. In the 
same figure we also show the validity of Eq. (f2"3"]) for Wt 
at large times and W[ at any time (see the difference be- 
tween empty and gray circles), as well as the asymptotic 
convergence to the large deviation rate function. 

The overdamped limit. — In the overdamped limit, 
the role of main variable is played by the tracer position 
x, while velocity v is neglected. Even if the dynamics is 
changed, and drift terms have different symmetries with 
respect to time-reversal, the final result appears identical, 
making robust our observation. 

The overdamped limit of Eq. (|24[) is a generalized 
Langevin equation with memory for the tracer position: 



70a; 



dU (x) 
etc 



+ F nc {x)+ 



M 

E 



" x{t')dt' + r](t) (39) 



with the same properties as in Eq. (|26|) for the noise rj(t) 
With a partial integration it is possible to cast Eq 
into 



7o a; 



dU (x) 



dx 



+ F nc {x) 



M 

^^X+ 



i=l 



M 



~x{t')dt' +v(t). (40) 



T»_/1 
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We use now the following auxiliary variables, with i € 
[1,M]: 



Xi(t) = — 



X(t')+Tij^i(t') \dt', (41) 



obtaining an equivalence with a system of Langevin equa- 
tions without memory of the form ([7]) , with N = M + 1 
(identifying Xq = x and Xj = Xi), and with 



7o -Do 
D, 



dUp(x) 
dx 



M 



i = l 



-(x-Xi) (i>0) 



F nc {x)-Y^^ipz-Xi) (42) 
Ti 



D i:j = 5 i3 - 



(43) 



Following the same procedure of identification of ir- 
reversible and reversible parts of drifts, and recognizing 
again that memory auxiliary variables have parity = 1, 
we obtain 



D: ev = 0, Df = D, 



(44) 



This leads to identifying, for the entropy production, the 
following expression, after suitable partial integrations: 



W, 



J i- |f„ c [x(s)] + 

M >| 

+ ^Fi[x{8)] \x(s)ds (45) 



; = 1 



where 



Ui(t) = £-[x(t) - Xl (t)f (i>0) (46) 

ZTi 

F i = f i (l-^j[x i (s)-x(s)}. (47) 

As seen, also for the overdamped case, entropy produc- 
tion is equivalent to the work done by the external non- 
conservative force plus non-conservative forces i*i(s) due 
to memory. It is easy to verify, through a partial integra- 
tion and the comparison between Eq. ((27|) and Eq. (j4"Tjl . 
that 



7i 



[Xi(t)-X(t)] =-J±Vi(t), 



(48) 



and therefore the forces in Eq. (|47|) are exactly equiva- 
lent to the forces in Eq. (|36|) for the inertial dynamics. 
Again we have verified, through numerical integration of 
Eq. ([3^1) . that the pdf of W t for large times reproduces 
the FR. For short times, the terms b t due to the steady 
state invariant measure at initial and final configurations 
must be added in order to recover the short-times kind of 
symmetry. The same consideration drawn for the inertial 
dynamics can be repeated here: when Ti = T, for all i, i.e. 
when the FDR of the second kind holds, then Fi = and 
the boundary term bt cancels out the exact differences in 
Eq. (|45|) . so that, if F nc = 0, detailed balance holds. 



In conclusion, the only difference between expres- 
sion ([35]) and (|45|) for the entropy production, is given 
by exact differences: these differences, for large times, can 
be neglected and the two expressions become equivalent. 
This is coherent with the fact that the overdamped dy- 
namics ignores the short time-scale corresponding to the 
relaxation of velocity. 

Linear response. — The first-kind FDR for the over- 
damped model in (|39p has been treated in several papers, 
see for instance [12, 16]. More in general, it is known that 
a generalized FDR [8,11,29-31] is satisfied if all variables 
X = Xq , X\ , . . . , Xm are taken into account with their 
steady state invariant measure /(X), provided that it is 
smooth and non vanishing, and the system is mixing. 

If an impulsive variation of coordinates {6Xi(0)} at time 
is considered, the generalized FDR for the response takes 
the form 



B, 



d g SX 3 (t) 
' 6Xi(0) 
91og/(X 



(^•(t)fii(0)) 



dX, 



(49) 
(50) 



where we use 7 to mean non-equilibrium averages follow- 
ing the perturbation at time 0, while (•) represents an 
ensemble average, which (under ergodicity) is equivalent 
to averaging over a long trajectory in the stationary state. 
If the diffusion matrix, Dij , is diagonal, one has [20] : 



B% = Bf + B* 
B? = l> / IK 



Bt 



Duf 



(51) 
(52) 



When detailed balance is satisfied, which implies that 
Wt = on average and W[ = for each trajectory, one 
has Sf = [20] and therefore 



(53) 



It is straightforward to verify that Eq. (|5"3"|) takes standard 
equilibrium forms, e.g. it is equivalent to Eq. ^ for ve- 
locity, or more in general to Kubo relations [7,20], which 
depends upon the choice of system, perturbation and 
measured responses. As discussed in previous sections, 
for models (j2"4")l and (f3"9"]l . when external non-conservative 
forces are absent, detailed balance condition Sf = cor- 
responds to Ti = To for all i , i.e. to the FDR of the second 
kind, Eq. ©. 

On the contrary, when detailed balance is not satisfied, 
the response includes an additive contribution: 



R.^Rl + iX^Bim 



Interestingly, comparison of Eqs. 
gives the following identification: 



(54) 

(IHI), (J2U) and (J521) 

(55) 
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This relation illustrates the connection between the "vio- 
lation" of Eq. (|53p and a part of the entropy production, 
o\. Indeed, relation (|55[) becomes very useful in the over- 
damped dynamics, where &i = 0: in this case the gener- 
alized force B* which entirely contributes to the entropy 
production a' = <7x, is the same force acting as "conju- 
gate quantity" in the additional (non-equilibrium) terms 
of the linear response formula. In the case with inertia, 
relation (|55|) becomes less useful, since o\ only contains 
exact differential and the bulk contribution to the entropy 
production comes from 02. 

We wish to point out that other approaches toward the 
connection between FDR and entropy production have 
also been discussed, from different points of view, in 
many recent works, e.g. [32-35]: for a detailed review, 
see also [36]. 

Conclusions. — Summarizing, we have discussed the 
role of memory in non-equilibrium steady states, merg- 
ing two main observations: 1) a general formula for en- 
tropy production in multivariate memory-less Langevin 
processes and 2) the mapping between Langevin models 
with short-ranged memory toward memory-less Langevin 
models with auxiliary variables. The latter leads to iden- 
tify memory as a non-conservative force. These forces 
cease to contribute to entropy production only under the 
validity of the FDR of the second kind, Eq. ([5]). 

Interestingly, in the overdamped dynamics, these same 
forces contribute to the so-called "violations" of the FDR 
of the first kind, when detailed balance is not satisfied. 
Future work should include a generalization to Langevin 
equations with other forms of memory. Investigation of 
the case with inertia is also needed, to better explore the 
connections with the linear response theory. 
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